DOI: 10.15415/jotitt.2014.22011 


Indian Sign Language Recognition System for 
Differently-able People 

ER.KANIKA GOYAL 1 , AMITOJ SINGH 2 

'Me Fellowship (cse), "Assistant Research Director, Chitkara University, Punjab, India 

Email: kanika.goyal @ chitkara.edu.in 

Chitkara University, Punjab 

Received: September 10, 2014| Revised: October 12, 2014|Accepted: November 20, 2014 
Published online: December 30, 2014 

The Author(s) 2014. This article is published with open access at www.chitkara.edu.in/publications 

Abstract: Sign languages commonly develop in deaf communities, that can 
include interpreters and friends and families of deaf people as well as people 
who are deaf or hard of hearing themselves. Sign Language Recognition is one 
of the most growing fields of research today. There are Many new techniques 
that have been developed recently in these fields. Here in this paper, we 
will propose a system for conversion of Indian sign language to text using 
Open CV. 

OpenCV designed to generate motion template images that can be used 
to rapidly determine where that motion occurred, how that motion occurred, 
and in which direction it occurred. There is also support for static gesture 
recognition in OpenCV which can locate hand position and define orientation 
(right or left) in image and create hand mask image. In this we will use image 
processing in which captured image will be processed which are digital in 
nature by the digital computer. By this we will enhance the quality of a picture 
so that it looks better. 

Our aim is to design a human computer interface system that can recognize 
language of the deaf and dumb accurately. 
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1. INTRODUCTION 

One of the most precious gifts of nature to the human breed is the ability to 
express himself by responding to the events occurring in his surroundings. 
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Every normal person sees, listens and then reacts to situations by speaking 
himself out in the environment. But there are some less fortunate people those 
are deprived of many valuable gift. Such impaired people rely on some sort 
of sign language for communicating their feelings to others. The deaf, dumb 
and the blind follow same problems or issues, when it comes to the use of 
computers. 

In the era of advanced technology, where computers or laptops and other 
processor based devices are an integral part of day to day life, efforts are 
required to be done for making the disables more independent in life. Inspite 
of that, there are people who are less fortunate than us and are physically i, 
may it be deafness or being aphonic. Such people lag behind impaired, their 
non-handicapped peers in using these technologies. These people have some 
expectations from the researchers and mostly from a computer scientist that 
we, computer scientists can provide some machine/model which help them to 
communicate and express their feelings with others. 

Very few researchers’ have them in mind and provid their continuous works 
for such people. Communication is th most important part of life. Around 1% 
of the total populatio of the world is suffering from hearing impairment, and 
thei life is not as simple and easy as it is for human withou limitations.. 

Finding an experienced and qualified interpreter ever time is a difficult 
work and also unaffordable. Moreover people who are not impaired, never try 
to learn sign languag for interacting with the impaired people. This becomes a 
caus of isolation of the impaired people. But if the system can b programmed 
in such a way that it can translate sign languag to text, the difference between 
the normal people and th impaired people can be reduced. 

Closely related with image processing are compute graphics and computer 
vision. In computer graphics, image are processed from physical models of 
objects and lighting instead of being acquired (via imaging devices such as vide 
cam, cameras) from natural scenes, as like in most animate movies. Computer 
vision, on the other hand, is considered a the high-level image processing out 
of which machine/computer/software intends to decipher the physica contents 
of an image or a sequence of images (e.g., videos o 3D full-body magnetic 
resonance scans). 


2. LITERATURE REVIEW 

Many Different approaches have been used by different researchers for 
recognition of different hand gestures which were implemented in different 
fields, few of the approaches were vision based approaches, soft computing 
approaches like Artificial Neural Network, Genetic Algorithm, Fuzzy logic, 
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data glove based approaches, and others like Canonical Analysis, PCA etc. 
All approaches could be divided into three categories- Feature extraction 
approaches, Fland segmentation approaches and Gesture recognition 
approaches. Few of the appraoches have been discussed in this paper. 

Many researchers [1-11] used skin filtering technique for segmentation of 
hand. This technique separates non-skin colored pixels from the skin colored 
pixels, thus extracting the hand from the background image. 

Fang [12] used Adaptive Boost algorithm which could not only detect 
single hand but also the overlapped hands. In [13-15] external aid like color 
gloves, data gloves were used by the researchers for segmentation purpose. 
In [1] [ 16-18] Principal Component Analysis (PCA) was used for extracting 
features for recognition of various hand gestures. 

3. THEORETICAL BACKGROUND 

OpenCV (Open Source Computer Vision Library) is an open source computer 
vision and machine learning software library. It was build to provide a common 
infrastructure for computer vision applications and to accelerate the use of 
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machine perception in commercial products. As it a BSD-licensed product, 
OpenCV makes it easy for businesses to utilize and modify the code. 

OpenCV leans mostly towards real-time vision applications. OpenCV is 
written natively in C++ and has a templated interface that works seamlessly 
with STL containers. 

The proposed algorithm consisted of four major steps which are namely 
Image Acquisition, Feature Extraction, orientation detection and Gesture 
Recognition which is also shown in the below given Fig 


OpenCV Overview: > 500 functions 

l- . . 

fife General Image Processing Functions 

III 


HI 


Ipport 


Image Pyramids 



4. METHODOLOGY 
4.1 Image Acquisition 

The first step of Image Acquisition is of acquiring an image during runtime 
through integrated camera and while acquiring these images will be stored 
in the directory after they are captured and the recently captured image will 
be acquired and that image will be compared with images stored for specific 
letter in the database using the SIFT algorithm and the comparison will give 
the gesture that was done and the translated text for the following gesture. The 
images will be captured through basic code of opening a web cam through 
OPENCV and then capturing the image through frames per second which will 
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be stored in another directory where all the inputs images are stored in another 
directory and the recent captured image is picked up and the comparison with 
given set of images are made. 

4.2 Feature Extraction 

For any of the object there are many features, interesting points on the object, 
which can be extracted to provide a “feature” description of the object. SIFT 
image features gives a set of features of an object which are not affected by 
many of the complications experienced in other methods, like object scaling 
and rotation. SIFT approach, for generation of image feature, takes a picture 
and transform it into a “big collection of local feature vectors”. Each of the 
feature vectors never changes to any of scaling, rotation or translation of the 
image. To extract such features the SIFT algorithm applies a 4 stage filtering 
approach [19] 
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4.3 Orientation Detection 

In it, will take the input of hand movement in any of the form or any orientation 
the gesture will be detected by the described section of feature extraction as the 
SIFT algorithm also includes the orientation assignment procedure. 

4.4. Gesture Recognition 

Finally when the whole process is complete the application will be then 
converted into its recognized character or alphabet from the gesture which 
might be helpful to be understood in layman’s language. The following process 
contain passing out the 1- dimensional array of 26 character corresponding 
to alphabets has been passed where the image number stored in database is 
provided in the array. 


5. CONCLUSION 

The system will provide an interface that can easily communicate with deaf 
people by Sign Language Recognition. The system is not only can apply in 
family environment, but also can apply in public. For the Social use these 
system is very helpful for deaf and dumb people. 

We will build simple gesture recognizer based on OpenCv toolkit and 
integrated it into Visionary framework. As a yes gesture we will mark up and 
down hand motions no matter which hand is used. 
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